00100		In the interest of having an efficient transfer of
00200	information to me, I would like to suggest some guidelines
00300	for the discussion of the research program.  First, I would
00400	like a top-down, breadth-first presentation of the objectives,
00500	associating with each top-level objective the
00600	approach to achievement, the identification of the people
00700	working on it, and an estimate of the cost (per year and
00800	over-all).  Each objective should be broken down into
00900	a tree of subobjectives, set against a time scale.  For the
01000	past, accomplishments should be summarized, and, for the future,
01100	milestones should be laid out.  Having this information prepared
01200	ahead of time and in form that will let me take a copy with me
01300	would speed up the discussion and save me from taking notes.
01400	
01500		In the foregoing, objectives, accomplishments, and the
01600	like should be definite.  It is not a valid objective 'to explore
01700	x'.  If 'x' is a new continent, gold may be an objective, or
01800	reaching the peak of the highest mountain, or establishing
01900	trade with the natives at a given profit level and in a given amount.
02000	The same general semantics appears to apply equally to scientific
02100	and technical exploration.  In any event, in order to use the
02200	information I am going to have to get it into definite-objective,
02300	definite-cost, definite-accomplishment form, and it will help me
02400	if you will do some of the packaging for me.
02500	
02600		After the over-all shape of the program is defined,
02700	the discussion should subdivide itself into N subdiscussions,
02800	one for each main subproject, and each of these should start out
02900	with a very short top-down, breadth-first introduction---but then go
03000	into informal mode with interaction predominating.  I would like,
03100	in the subdiscussions, to get acquainted with the people doing
03200	the work---not everyone, of course, but one, two, or three people
03300	per subproject, depending on its size and importance.
03400	
03500		At the end of the visit, I would like to have at least half
03600	an hour with the Principal Investigator and any of his right-hand
03700	people he thinks appropriate.  This would be a kind of executive
03800	session.
     

00100	OBJECTIVE Make machines which are good tools for humans, particularly
00105	for  assembly:      machines  which  are  intelligent,  ie which SEE,
00110	MANIPULATE their environment, and REASON about how to manipulate.
00115	
00120				ROBOTICS
00125		MANIPULATION		REASONING		SEEING
00130		Language for Assembly	Assembly planning	Visual
00135	Feedback
00140		Sensing						Complex
00145	Objects
00150		Control using Sensors
00155		Synchrony of Manip
00160	
00165				PEOPLE
00170				Binford 1/2
00175		Finkel	free		Taylor	free				
00180						Miyamoto free		Bolles free
00185								Schukin
00190	student
00195		Takeda	free					Clancy
00200	student
00205								Ganapathy
00210	student
00215	
00220				others not supported
00225		Shimano	student					Pingle staff
00230		Scheinman student
00235	
00240	
00245			I.	MANIPULATION OBJECTIVE
00250		To provide a system which can be easily programmed to perform
00255	a  wide  variety  of  manipulation  tasks;  dexterous enough for many
00260	assemblies.
00265			SUB-OBJECTIVES
00270		Language for programmable assembly
00275		Sensing for manipulation
00280		Control based on sensing
00285		Synchronized motion of two manipulators
00290	
00295		LANGUAGE FOR PROGRAMMABLE ASSEMBLY
00300		We will implement  the  language  AL.   In  the  past,  Paul,
00305	Finkel,  Taylor,  and  Bolles have designed the language AL, based on
00310	Paul's  past  experience  with  WAVE,  our  previous   language   for
00315	manipulation.  It is our experience that WAVE has been very valuable.
00320	
00325		SENSING FOR MANIPULATION
00330		We will implement force sensors with sensitivity of  20  gms.
00335	We  will  implement  touch  sensors with sensitivity of 2gms on a 2x3
00340	matrix for each finger.   We will be able to  do  adaptive  grasping,
00345	exploring with touch, and inserting screws into holes by tipping them
00350	and feeling when they drop into the hole. Previously,  we  have  used
00355	force  sensing  based on motor torques with a sensitivity of 500 gms,
00360	and touch sensing based on 1 microswitch per finger with  sensitivity
00365	of 10 gms.
00370	
00375		CONTROL BASED ON TOUCH AND FORCE
00380		CONTROL OF SYNCHRONIZED MANIPULATORS
00385		We  will formulate the theoretical basis for force control of
00390	manipulators, determine how sensitive force control can be  with  the
00395	Scheinman  manipulators,  and  optimize  those  abilities and provide
00400	language  primitives.       We  will  analyze  and  experiment   with
00405	synchronized   control  of  two  manipulators,  to  provide  language
00410	primitives for AL. We will carry out tasks which require two arms  in
00415	synchrony,  sensitive touch, and force. A man can do a lot blind, but
00420	much less with one arm, only two fingers, and poor sense of force and
00425	touch. We were the first group to use two manipulators under computer
00430	control. In the past, Paul and Bolles have performed  assembly  tasks
00435	using  force  and touch sensing. We have formulated the force control
00440	and synchronization primitives which now are in AL.
00445	
00450	
00455			II.	REASONING ABOUT MANIPULATION	OBJECTIVE  To
00460	provide a system which programs many of the simple steps required  in
00465	performing   manipulations,   to   relieve   humans   of  detail  and
00470	bookkeeping.
00475	
00480			VERY HIGH LEVEL LANGUAGE FOR PROGRAMMABLE ASSEMBLY
00485		We   will   implement  a  system  which  aids  in  convenient
00490	programming of an assembly of the difficulty of a two-stroke gasoline
00495	engine, in commands at the level of a manual for assembly.  Imagine a
00500	production  engineer  setting  up  an  assembly  station.    We  will
00505	accomplish this by maintaining a planning model, including a powerful
00510	determination of locus of  objects  under  symbolic  constraints,  by
00515	modelling  objects,  and  operations,  by  testing  collision  of the
00520	manipulator with the assembly environment, and by choosing  sequences
00525	of operations that do not lead to awkward and wasteful or unnecessary
00530	motions.    We will provide the system with the ability  to  ask  for
00535	necessary  advice.   We will interface as much as possible with other
00540	work on automatic programming. Taylor has designed the planning model
00545	in AL, programmed a previous preliminary system to resolve sequencing
00550	problems, and made a package  to  determine  the  locus  of  objects.
00555	Luckham  has  programmed  the  sequence  of  the  pump  assembly (not
00560	including manipulation) in the Buchanan-Luckham automatic programming
00565	system.
00570	
00575				III.	VISION
00580		OBJECTIVE:     To  provide  a  system  which  can  be  easily
00585	programmed  to  see  a  useful class of scenes, specifically:  visual
00590	feedback for control of assembly; description and  identification  of
00595	complex objects in a complex visual environment; vision for planetary
00600	exploration.
00605	
00610		VISUAL FEEDBACK FOR CONTROL OF ASSEMBLY
00615		We  will  sense the positioning error of real parts, to guide
00620	their assembly.   They may be curved objects, shiny, dirty; they  are
00625	not  expected to be specially painted or prepared.  We are assuming a
00630	situation realistic for assembly; that the operation  is  repetitive,
00635	with  small  shifts  of  parts.   For example:  insert a screw into a
00640	distributor body with position  variation  of  1  cm;  or  align  two
00645	surfaces.  We  sense the relative location of features (determined by
00650	edge and small area correlation) in a single image, and in  pairs  of
00655	images.  We relate relative locations in images to relative locations
00660	in the world.   As tools, we  require  a  hand  held  camera,  stereo
00665	camera,  stereo  correspondence  techniques,  a  vision language, and
00670	representation of image models. Previous work has been  with  blocks,
00675	often  specially  painted for high contrast. Our past accomplishments
00680	are relative alignment of blocks using outlines; using internal edges
00685	of  specially  painted  blocks,  using  stereo  (multiple images from
00690	turntable); and most recently for blocks without special preparation.
00695	
00700			COMPLEX OBJECTS
00705		We  will  describe  the shape of and identify complex objects
00710	from a rich set of possible objects in a complex visual  environment,
00715	using  tv  images.  We  sense  depth  discontinuities using laser and
00720	stereo, and sense region boundaries  using  edge,  region  and  curve
00725	linking  techniques.    We  describe  region  boundaries using region
00730	description routines as used by Nevatia and Binford.  We relate these
00735	features  to objects using the object representation and 3d cues from
00740	2d.   As tools, we  require  stereo  cameras,  laser  ranging  setup,
00745	representation   and  vision  language.   Our  past  accomplishments:
00750	Binford designed a useful object  representation  and  laser  ranging
00755	technique;  Agin  built  ranging  device  and  implemented  system to
00760	describe  parts  of  objects;  Nevatia   greatly   generalized   part
00765	descriptions,  described  complete complete objects, recognized doll,
00770	toy horse, hammer.   We were the first to recognize objects  of  this
00775	complexity,  including  a  structured  visual  memory  which required
00780	matching against only a subclass of similar objects, as against brute
00785	force   searching   of  the  entire  set  of  models.  We  have  also
00790	experimented  extensively  with  motion  parallax,  and  with  region
00795	separation based on color and texture.
00800	
00805	
00810	
00815	
00820			ACCOMPLISHMENTS AND MILESTONES
00825	
00830				MANIPULATION
00835	
00840		LANGUAGE	SENSING		CONTROL		STRTEGIES	
00845	
00850										1972
00855	
00860	
00865	
00870										1973
00875	
00880	
00885	
00890	
00895										1974
00900	
00905	
00910	
00915	
00920										1975
00925	
00930	
00935	
00940	
00945										1976
00950	
00955	
00960	
00965	
00970									1977
00975	
00980	
00985	
00990	
00995										1978
     

00100	LANGUAGE FOR PROGRAMMABLE ASSEMBLY
00200	
00300	ADVANCED MANIPULATION CAPABILITIES
00400	  mini system
00500	  sensors: touch,force
00600	  formulate analysis and control language
00700	    touch: grasping and shape recognition
00800	    force control
00900	    synchronized simultaneous motion
01000	  task execution
01100	
01200	VERY HIGH LEVEL LANGUAGE FOR PROGRAMMABLE ASSEMBLY
01300	  representation
01400	  constraints-locus info
01500	  planning model
01600	  consistent sequences
01700	  collision detection
01800	
01900	
02000	
     

00100	OBJECTIVES
00200	programming of programmable devices at the level of manuals for assembly
00300	strategies and planning model for real world tasks
00400	accept advice, ask for advice, explain decisions to user
00500	facilities for integrating assembly with factory systems
00600	
00700	SUBOBJECTIVES
00800	provide a powerful language for accomplishing tasks by making a rich
00900	  semantics of models, repr, primitives
01000	Model what we know at each point in program, as much as needed for task
01100	  provide a powerful modelling language
01200	  ways in which user can use it
01300	Internal repr of knowledge
01400	Communicating with planning model
01500	
01600	MILESTONES
01700	March system
01800	  servo
01900	  primitive manipulation language
02000	  planning model simulation
02100	June system: level of pump program   OUTLINE SYSTEM
02200	  problem-solving paradigm
02300	  relational semantics
02400	  small library of parts feeding
02500	  primitives--peg in hole, fit face of one object to another face
02600	    not just canned strategies; flexible and modified
02700	  sequence tasks depending on estimates of awkwardness
02800	    don't work on backside; choose a good position initially to avoid unnecessary
02900	    motion and putting down
03000	  keep track of accuracy to select operations
03100	NEXT MILESTONE:  FILL IN SYSTEM  engine assembly in shop operations
03200	  expand operations
03300	  expand primitives to cover more cases
03400	  make planning model more sophisticated--world splitting, merging
03500	  handling contingencies using world model; planning for error recovery
03600	  planning for inspection
03700	  planning for vision
03800	  libraries and experimental work
03900	    code in library routines
04000	    represent library routines in machine readable form
04100	  OBJECT AND SHAPE REPRESENTATION
04200	  OBSTACLE AVOIDANCE
04300	  LOCUS INFO
04400	  GRAPHICS IN AL
04500	
04600	
04700	In what way is the domain interesting?
04800	  very rich semantics
04900	  modelling
05000	  economic significance
05100	  interaction with environment--plan for knowledge gathering, modelling influence
05200	    of knowledge on ther knowledge
05300	  interaction with human users
05400	
05500	
     

00100	We address these primary vision problems:
00200		1)  What  tasks  can be done with current computing capacity?
00300	PDP10 class machines have a million times less processing power  than
00400	what  we  know of the human low level (Hubel-Wiesel and stereo cells)
00500	visual system. We think a variety of tasks can be chosen.
00600		2.    What can be done with a million times current computing
00700	capacity and large memory? Special-purpose computers well within  the
00800	state   of   the   art  could  gain  a  factor  of  1000,  while  the
00900	semi-conductor  industry  is   making   revolutionary   advances   in
01000	complexity of devices, which could translate into parallelism for low
01100	level operations. In a few decades, gains of a million times  current
01200	computing  power  may  be  available.    If we had such machines, our
01300	understanding and software would not allow us to  make  full  use  of
01400	that  power,  although  we  could  make  significant  use  of current
01500	techniques which we avoid because of computation cost.
01600		3.  What representations make possible strategies and systems
01700	of reasonable complexity, systems which are programmable for  a  wide
01800	range of tasks? Programs are complex, even when they deal with only a
01900	small knowledge base and only one or two visual facilities, and  will
02000	be  much  more  complex  when they use the variety of possible visual
02100	facilities.  The problems are much larger than can be accomplished by
02200	a single researcher, or single laboratory.
02300	
02400	Visual  feedback  tasks  for  assembly,  such as putting a screw in a
02500	hole, are among those tasks  which  can  be  done  now  with  current
02600	hardware with reasonable time. This feedback facility has utility for
02700	those tasks for which positioning is sufficiently  variable  to  make
02800	vision  faster  than  blind  searching  using  force feedback, or for
02900	setting up programmable assembly  devices.        In  the  industrial
03000	environment,  the  tasks  are repetitive, the objects are well known.
03100	Objects have glare, shadows, textures and  curved  surfaces.    Since
03200	locations  of objects are reasonably well known, we want to determine
03300	small changes in  location.           A  sample  picture  is  a  good
03400	two-dimensional  model  for  subsequent images.     We are building a
03500	rapidly programmable system for a variety of visual  feedback  tasks.
03600	The  system  will  describe two-dimensional images in terms of curves
03700	and  regions,  interact  with  the  user  who  can  supply   symbolic
03800	information,  and  generate  effective  sequences  of  operations  to
03900	localize shapes in subsequent images, using curve  matching,  stereo,
04000	and correlation of small features.  Some features such as long curves
04100	require less time to find than small features, and the effective plan
04200	is graph-structured with near-minimum time and acceptable reliability
04300	and contingency.     This will be a major contribution to  strategies
04400	involving  structured  properties such as shape, as opposed to solely
04500	point properties such as color.
04600	
04700	We have succeeded in recognizing complex objects such  as  dolls  and
04800	toy  horses,  with  some  obscuration,  using depth perception with a
04900	laser.        That  work  demonstrated  the  concept  of  description
05000	determined  by  a  representation  of  three-dimensional shape.    An
05100	important feature  was  that  the  recognition  took  place,  not  by
05200	matching  against  a  list  of  all  models in the visual memory, but
05300	against a subset of similar objects,  obtained  by  indexing  into  a
05400	structured  visual  memory.    We will extend that project in several
05500	directions:   we will deal with more  severe  obscuration  by  making
05600	multiple  and more local shape indexing; use stereo and monocular tv;
05700	increase the distinguishing power by increasing the resolution of low
05800	level  shape  descriptions (some of the simplest identifying features
05900	are fine details, such as human hands) and increasing the  resolution
06000	using top-down verification; improve the concepts of body separation;
06100	use several levels of (and more global) description.
06200	
06300	Several threads run through the plan of our vision projects: they all
06400	use  stereo,  both  bulk  correlation and feature correlation.  Other
06500	modules  will  be  developed  and  contribute  across  the  range  of
06600	projects,  including  a  curve linking process for curved objects and
06700	visual feedback (extending the ellipse finding  routine  of  Perkins,
06800	used  to  recognize  cylinders), and an optimal curve boundary scheme
06900	for region determination.
07000	
07100	Both visual feedback and the experiments with  complex  objects  will
07200	involve a strategy phase.  In our system, there will be both top-down
07300	(recognition) strategies,  and  bottom-up  (description)  strategies.
07400	Our   ccntention   is  that  vision  involves  extensive  description
07500	facilities in addition to recognition.   The  description  strategies
07600	are the more difficult.
07700	
07800	We  will  base  our  strategies  on  representation  of  objects  and
07900	intermediate control structures.  These representations  provide  the
08000	semantics  of  a  language  in  which  to express strategies, and the
08100	structure of programs.  Just as formalization of  syntax  provided  a
08200	basis  for  systematizing  compilation of complex syntax, formalizing
08300	semantics  enables  systematizing   compilation   of   complex   data
08400	structures.   This  vision  language  is  embedded  in  SAIL; it is a
08500	language in the  sense  of  providing  a  rich  semantics  of  visual
08600	structures.
08700	
08800	
08900	
09000	
     

00100			MANIPULATION
00200	
00300	
00400	We propose to investigate intelligent systems which use programmable
00500	manipulators to interact with their environment.  We have chosen the
00600	paradigm of automated assembly.  We have established leadership in
00700	manipulation by a coherent program: designing and building high quality
00800	manipulators [] (which several labs have copied); developing software servo 
00900	force and touch control []; developing a language for manipulation, WAVE [].
01000	We have carried out several assembly tasks [] involving force and touch,
01100	vision, tools and two arms.  Use of the language WAVE made possible a degree of
01200	generality; we evolved a library of macros of assembly modules which made successive
01300	tasks increasingly quick and easy to program.  Of course, the system developed
01400	for assembly is applicable to other domains, but choosing a particular domain has 
01500	kept our plan focussed.  We have also developed a nucleus of talented researchers.
01600	
01700	We plan the following projects:
01800	a) language for programmable assembly
01900	b) advanced manipulation abilities
02000	c) very high level language for programmable assembly.
02100	We give details below.
02200	
02300		LANGUAGE FOR PROGRAMMABLE ASSEMBLY
02400	We have designed a language for programmable assembly, to succeed WAVE.  Our
02500	past experience is that WAVE, our previous manipulation language, has been
02600	extremely valuable.  The new language is high level, with special specification
02700	of trajectories, force control, monitors for conditions such as touch, and
02800	synchronization of two or more manipulators.  The language also is model-based
02900	in that it keeps a planning model of the environment for compile time evaluation.
03000	We describe planning facilities under the section on very high level language
03100	for programmable assembly.  The language can control a wide variety of devices
03200	by including special solution programs for each, and the framework is adequate
03300	to control more than two devices.  AL is a research vehicle.  Some interesting
03400	concepts for synchronizing manipulators, and force control have been included in the language, but we feel
03500	language, but we believe
03600	that these a priori designs are inadequate.  Analytic studies and experiments will
03700	improve our understanding and language abilities for control of force and synchronized
03800	manipulators
03900	
04000		ADVANCED MANIPULATION FACILITIES
04100	We have accomplished assembly tasks with crude force and touch sensing.  The system
04200	sensed motor currents of arm joint motors to sense forces greater than 500 gms.
04300	A single microswitch on each finger sensed touch of more than 10 gms.  We are
04400	developing new sensor hardware: a wrist sensitive to torques and forces greater than
04500	20 gms; touch sensors with sensitivity 1 gm in a 2 by 3 matrix on each finger.
04600	
04700	
     

00100	what needed? what preconditions? When possible to be done?
00200	PROJECTS
00300	NSF
00400	finish AL			visual feedback (rcb,kkp,bgb)
00500	force (bes)			stereo (kkp,pdq,bgb)
00600	collision detect (bgb,kkp)	visual strategies & models (tob,ys)
00700	strategies assembly (rht)	vision language (tob,kkp)
00800	simultaneous motion (??)	touch (st,cj)
00900	gedanken expts (tob,..)		visual hardware
01000	descriptors (tob)		complex objects (??)
01100	task execution (bes,rht)
01200	
01300	FINISH 'AL: (RF,RHT,BES)
01400	runtime system (BES)
01500	compiler (RF,RHT)
01600	monitor (RF,RHT)
01700	
01800	FORCE SENSING (BES)
01900	sensor protection design (VDS)
02000	fabricate protection (VDS)
02100	interface w/ blue arm (TAG)---wait until arm interface debugged
02200	duplicate sensor (BES)
02300	runtime force calc (BES)
02400	force servo program (BES)  determine limits of force servo resolution
02500	conceptualization of force control (BES)
02600	
02700	COLLISION DETECTION (BGB,KKP,MIYAMOTO)
02800	algorithm
02900	graphics
03000	interface with AL
03100	
03200	ASSEMBLY STRATEGIES (RHT)
03300	manipulation expts
03400	maintain planning model
03500	locus info
03600	represent objects
03700	progressive refinements: specialist, bureaucrats
03800	
03900	SIMULTANEOUS MOTION OF TWO ARMS
04000	choose tasks
04100	program before AL ready
04200	help put features into AL
04300	test programs
04400	formulate new treatment
04500	
04600	GEDANKEN EXPTS: inspection, vision strategies, assembly
04700	collect those we have now (pump, gas engine)
04800	collect those of others
04900	survey industry
05000	work out other tasks
05100	summarize
05200	
05300	DESCRIPTORS: shape,assembly,fabrication,collision detect,strategies,
05400	   control structure,tasks
05500	overall scheme
05600	summarize
05700	survey industry
05800	detailed development
05900	summarize
06000	
06100	TASK EXECUTION
06200	force
06300	simultaneous motion
06400	test AL
06500	
06600	
06700	
06800	VISUAL FEEDBACK (RCB,KKP)
06900	2D repr
07000	interactive cursor system
07100	characterize curves
07200	cost-benefit analysis
07300	tolerance analysis-successive localization
07400	curve verification
07500	correlation
07600	minimization-graph search
07700	3d graphics
07800	
07900	VISUAL STRATEGIES
08000	primitives from Vision Language
08100	use graph primitives from AL
08200	
08300	VISION
08400	optimal curve
08500	stereo
08600	texture
08700	curve linking
08800	coarse sampling regions
08900	strategies-representation of structure
09000	representation of complex objects
09100	tasks-paradigms
09200	
09300	OPTIMAL CURVE
09400	local approx to intensity surface
09500	   sin(cosα*x+sinα*y+β); sinx,cosx siny,cosy
09600	extended approx consistent at interface
09700	analtyic form of local and extended forms
09800	computational complexity; limit of support; sensitivity
09900	
10000	TEXTURE
10100	region boundaries
10200	region descriptors or boundary descriptors
10300	proximity
10400	
10500	STEREO
10600	calibration-stereo coordinates
10700	boundaries
10800	context
10900	curved objects--envelope
11000	
11100	CURVE LINKING
11200	proximity angle, position
11300	calc limits on curve
11400	curve testing-coarse
11500	curve description-fine
11600	
11700	COMPLEX OBJECTS
11800	increase resolution, low level and top-down verify
11900	shape indexing
12000	stereo and monocular tv
12100	body separation
12200	cross scan linking using 2d array
12300	cross sections
12400	more global description
12500	description of junctions, cross section shape and articualtion inference
12600	obscuration
12700	
12800	
12900	TASKS****
13000	VISUAL FEEDBACK--2D--VERIFICATION VISION
13100	2d version
13200	speed-sacrifice generality
13300	  leads to special case kludges or strategy generation of such kludges
13400	programmability
13500	boundary descrition-quick characterizations
13600	  mark tools and hand
13700	  long curves--find a crossing easily
13800	  coarse region segmentation (characterize region from long curves)
13900	correlation for small feature windows
14000	
14100	FINDING OBJECTS AMONG A SMALL CLASS
14200	FIND TOOLS ON A CLUTTERED TABLE
14300	3d model
14400	feature abstraction for recognition
14500	structure of features
14600	bottom-up or top-down
14700	stereo-shape indexing
14800	
14900	
     

00100	FUNDAMENTAL VISION PROBLEMS
00200	segmentation-body,part
00300	obscuration-indexing
00400	representation
00500	computational complexity
00600	linking-indexing
00700	description vs recognition
00800	use of context
00900	learning
01000	low level, bottom up vs high level, top down
01100	
01200	COMPLEX OBJECTS
01300	small details, particularly terminations of limbs are very important
01400	in distinguishing similar objects (two animals); consider hands and
01500	feet of humans; thus, increase resolution at low level, use verification
01600	from high level; use fovea
01700	
01800	match on cross section by cross section basis. 
01900	hypothesis-verification techniques and cross section description
02000	
02100	boundary cross scan linking for laser by 2d arrays (perhaps two scales)
02200	
02300	analytic breakeven point in space between hash coding and nxm array.
02400	ie how many bits necessary in hash code entry.  What is the time cost?
02500	
02600	More sophisticate body separation techniques
02700	
02800	inference of articulation
02900	
03000	fore/aft symmetry--man
03100	
03200	more global choice of local descriptions--different sort of piece
03300	  axis finding; redescription-pinch off smaller pieces, extend larger
03400	  pieces
03500	
03600	is it possible to orient in a description: main piece
03700	  front/back; head/tail
03800	  head is like head of an arrow; supports are at the root end of a tree;
03900	  ie, head is single, feet are multiple--eg tripod, man
04000	
04100	straight axis pieces
04200	
04300	difficulties:
04400	widths depend on viewing angle
04500	lengths sensitive to angles of limbs, where make the discontinuity break
04600	
04700	
04800	intuitive distinction horse vs human
04900	upright vs all fours--angles of joints
05000	neck and head--short vs long and conical w/ bend at head
05100	width of legs vs width of body
05200	tail
05300	hands and feet
05400	shape of legs-horse more conical
05500	skin texture
05600	range of articulation--reject for now-not easily inferred from image
05700	  types of joints-ball and socket.  should be able to guess range of
05800	  some joints from image; ie arms vs legs; joints which conserve cross
05900	  section;  often, legs do not have wide range of opening along line
06000	  of pairing; legs are ball joints; knees also?
06100	
06200	size hierarchy and indexing
06300	
06400	
06500	REPRESENTATION 
06600	CAM tasks
06700	future: planning, predicting future performance
06800	present: information,design,orders,scheduling
06900	past: monitor performance
07000	
07100	dynamic characterization of devices in terms of self determination
07200	of torque, friction, gravity, acceleration terms for arm
07300	
07400	calculate volume moments, display for spine-cross sections
07500	
07600	represent fabrication operations
07700	represent assembly operations
07800	formalization of cylindrical transform
07900	spline fits to local coes from projection
08000	
08100	vision language
08200	symbolic entity
08300	spatial entity
08400	image entity
08500	operations
08600	  spatial-spatial   eg modelling, hypotheses
08700	  spatial-image     eg display
08800	  image-image       eg low level operators
08900	  image-spatial     eg 3d cues and hypotheses
09000	
09100	representation based on task
09200	tasks for descriptors
09300	  display
09400	    intersections
09500	    local visibility
09600	    global visibility
09700	    run-length coding w/ line to line coherence
09800	  mathematics-algebraic geometry
09900	    singularities
10000	    behavior of families
10100	  recognition
10200	  design
10300	  scheduling
10400	  optimization and costing
10500	  feasibility and error analysis
10600	  trajectory optimization of arms
10700	    utilization of expensive machines
10800	  machining
10900	    obstacle avoidance
11000	  numericallly controlled inspection and measurement
11100	  error and constraint satisfaction
11200	  monitoring and disaster detection
11300	tasks for design
11400	  display
11500	    intersections
11600	    local visibility
11700	    global visibility
11800	    run-length coding w/ line to line coherence
11900	  machining (fabrication operation planning)
12000	  assembly strategy
12100	  avoidance
12200	  visual perception
12300	  deciding similarity of shapes (in terms of fabrication operations)
12400	  constraint solution
12500	design elements--fabrication operations
12600	  holes (tapped), fillets, bosses, bevels, chamfer, countersink
12700	
12800	machining
12900	represent:
13000	  parts and material (blank)
13100	  lathe motions (degrees of freedom)
13200	  cutting tools, cutting dynamics and tool wear
13300	  constraints-collision avoidance-feed direction
13400	  dynamics-feed rates, torque, chatter
13500	  cost-tool wear, time for tool change, risk of breakage, rough and fine
13600	  errors-tolerance, roughness
13700	  feedback (machine sensors)
13800	  interaction with user
13900	tasks:
14000	  collision avoid
14100	  cost minimization
14200	  error satisfaction
14300	  monitoring and inspection
14400	  disaster detection
14500	  tool selection and clamping, programming output format
14600	  machine control
14700	  translate 3d geometry to 1d geometry
14800	  interact w/ scheduling
14900	operations
15000	  roughing
15100	  fine cut
15200	  grooving
15300	  drilling
15400	  threading
15500	  adaptive control/optimization
15600	time
15700	causal relations
15800	function
15900	structure
16000	environment
16100	pathology
16200	operations
16300	
16400	Barrow, Tenenbaum graph scheme is essentially a one-level representation;
16500	ie the depth data are essentially matched to the semantic representation
16600	(spatial relations).  A better system must be multi-level representaion.
16700	That one depends heavily on the assignment of local interpretations to features;
16800	I think that works only when the local features are shape features, which
16900	is straightforward with depth information, but not with image info.
17000	
17100	Waltz-like graph with general evaluation functions at nodes.  When a value
17200	changes, only nodes dependent on a value are changed.
17300	representation of castings
17400	applications: costing, simulation, quick feedback, resolve interrelationships
17500	deformable objects: separate objects with constraints
17600	information types, information flow, data structures in industry
17700	sensors available, control functions, what changable with better
17800	  sensors?
17900	geometrical specs, technological data, self-documentation and self description
18000	  of system to use in strategy; statistical operational data
18100	models for programming; strategy; costing and optimization-statistical;
18200	  interfacing to other devices; control and scheduling; internal execution
18300	
18400	current state of software does not allow modular systems, structured
18500	programming, definition of data types, symbolic reference, large address
18600	spaces, interaction
18700	It is supposed that business will use a subset of the system requirements
18800	of computing centers.  In actuality, the tasks are much more varied, more involved
18900	with  real-time and device control and more resemble AI environments than
19000	business and computing center (number-crunching) environments.
19100	
19200	relevance of strategies to statistical control and optimization, simulation
19300	
19400	characterizing function: parts, subassemblies, fasteners
19500	form is function
19600	
19700	We do not think a theory as such is possible for anything as
19800	complex as manufacture or assembly.  Instead, we aim to provide a languag
19900	with which to communicate, control.  Make the system hierarchy like the
20000	cost hierarchy.
20100	
20200	Constraints imply force.  These may be inequalities, limited by some
20300	disaster (breaking).  These can be geometric constraints, but if
20400	dynamics are important, force conditions are useful.
20500	
20600	Similarity between hardware/software architecture and industrial
20700	production systems: 
20800	pipelining vs parallelism for speed
20900	serial execution means speed limited by slowest operation
21000	  reliability limited by product of reliabilities of individual elements
21100	synchronous vs asynchronous
21200	why have industrial systems emphasized one structure of system?
21300	  apparently industry has chosen the pipelining serial form of
21400	  system as opposed to parallel.  This is in spite of the facts
21500	  that:
21600	    it is easier to make changes in a parallel system; change one
21700	      machine at a time, leaving bulk of system operting
21800	    reliability is higher; one machine down leaves production intact
21900	    speed is higher; different operations take different times
22000	      instead of all operations take time of slowest
22100	apparently, the answer is that parallel systems require redundant
22200	  hardware; if there are n parallel stations, then there must be
22300	  n feeders for each stage, n jigs, etc.
22400	another reason is that the assembly line is justified by simplifying
22500	  training of human labor
22600	Descriptor systems are the basis for success of large systems which
22700	require coordination of the efforts of several/many people, and the
22800	basis for pessimism about large scale efforts is the lack of such
22900	descriptor systems
23000	
23100	Curve descriptors: interpolation of tangent vectors
23200	(p1,v1) to (p2,v2) cannot be linear since v1 may equal v2 and may
23300	not lie along p2-p1.
23400	We can break down into velocity components normal to p2-p1 and parallel
23500	to p2-p1.  Application to arm interpolation.
23600	F=m*dv/dt;   et=(p2-p1)/|p2-p1|
23700	Ft=(F.et)  tangential component;  Fn=F-Ft*et  normal force vector
23800	
23900	Subsidiary condition for translational invariance
24000	  the surface is (in some sense defined below) aligned along the normal
24100	  to the cutting plane; in 2d, the axis is the bisector of the line and
24200	  bisector of angles;  in general for locally ruled surfaces, the
24300	  "generating lines"=directions of minimum curvature; vectors normal to
24400	  in cutting plane and in tangent plane; components of vectors in plane
24500	  should add to zero.  We need some math results here.
24600	Need representation for force and torque control of manipulation
24700	  Whitney's feedback matrix is ad hoc; he presents no model or way
24800	  of calculating the matrix;  fiddling around with the feedback
24900	  matrix raises serious questions about control stability
25000	
25100	Classification of tasks in terms of range of manipulation (is locomotion
25200	  necessary) is just one way of classification
25300	
25400	statistics, trend analysis
25500	
25600	avoidance, collision detection
25700	
25800	summarize state of art in devices, sensors, software
25900	
26000	assembly tasks in force vector sense
26100	
26200	Waltz-like graph relaxation scheme
26300	
26400	characterizing function of parts, subassemblies
26500	
26600	interpreter for description network
26700	
26800	symbolic vector geometry package
26900	
27000	tolerancing program
27100	
27200	methods of attachment
27300	  insertion into hole; investigate intersection
27400	
27500	lattice world model w/ negation, conjunction, disjunction, conditionals
27600	  must have, might have, must not, might not
27700	
27800	projection multi-entry spaces for avoidance of elongated objects
27900	
28000	articulations- away from interference
28100	  hips-motion is away from other limb, along body
28200	  hand likewise
28300	
28400	Turner's thesis--Huffman-Waltz vertex rules w/ curved objects
28500	REGIONS
28600	the more precise the statistical question asked, the more restrictive
28700	(fewer combinatorial possibilities of random occurence) and the more
28800	sensitive is the test.
28900	
29000	multi-level edge finding: make local descriptions over a grid
29100	(like on 2x2 centers)  Then make more global descriptions along
29200	curves.  Descriptions combine the local descriptions into optimal
29300	smooth curve schemes.
29400	
29500	
29600	VISUAL FEEDBACK
29700	mark tools and arm
29800	
29900	JUSTIFICATIONS FOR PROGRAMMABLE ASSEMBLY
30000	incorporate experience better, more reproducibly, systematically
30100	shortage of highly skilled labor in industry.  Army feels it.
30200	eliminating or automating production of jigs and fixtures (prototyping shop)
30300	  programmable jigs; simple arms
30400	cut down on inventory--store tapes
30500	incremental steps from hard automation--what are they?
30600	
30700	VISION HARDWARE
30800	silicon vidicon advantages
30900	  more sensitive--lower laser power requirements
31000	    increase depth of field
31100	  lower blooming--narrower laser traces
31200	    more flexibility for ordinary tv uses
31300	  less lag--faster laser reading
31400	    allows motion, no burn-in
31500	  higher signal current--higher signal to noise
31600	disadvantages
31700	  variations in sensitivity--how big are they?
31800	  no control on sensitivity--that's ok
31900	
32000	
32100	
32200	COMPANY CONTACTS
32300	Teledyne-Ryan: David A. Carnevale
32400	Texas Instruments: Cliff Penn
32500	AMP
32600	Bendix
32700	Whirlpool
32800	Voelker
32900	Nevins
33000	GM-Perkins, GM application study for vision
33100	IBM-Will
33200	Northrup-Gainnie